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Abstract 



The material in this note is used as an introduction to distributed 
algorithms in a four year course on software and automatic control system 
in the computer technology department of the Komsomolsk-on- Amur state 
technical university. All our the program examples are written in Borland 
C/C++ 5.02 for Windows 95/98/2000/NT/XP, and hence suit to compile 
and execute by Visual C/CH — h We consider the following approaches of 
the distributed computing: 

The conversion of recursive algorithms to multithread applications 

A realization of the pairing algorithm 

The building of wave systems by Petri nets and object oriented pro- 
gramming. 
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1 Threads and the conversion of recursive pro- 
grams 

A thread is declared by its descriptor. We create the thread by the API function 

HANDLE h=CreateThread (NULL, 0, Thr, (void *)p, 0, 0); 

which loads the subroutine 

DWORD WINAPI Thr (void *p) 
{ 



and returns the descriptor h of the thread if the creation is successful and 
h=NULL otherwise. The argument p points to a list of parameters of the 
function Thr(). The subroutine call 



WaitForSingleObject(h, INFINITE) ; 
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leads to the waiting of the thread termination. 

If a program has a recursive function, then we can convert all the function 
calls to the thread creations. This conversion leads to a multithread application. 
We consider the Hoare quick sort for example: 

#include <stdlib.h> 
#include <time.h> 
#include <iostream.h> 
#define N 100 
int x [N] ; 

void q_sort(int 1, int u) 
{ 

int i, j, m, t; int temp; 
if(l < u) 
{ 

t = x [1] ; m = 1 ; 

for (i = 1 + 1; i <= u; i++) 

if(x[i] < t) 

{ 

m++; temp = x[m]; x[m] = x[i]; x[i] = temp; 

} 

temp = x[l]; x[l] = x [m] ; x [m] = temp; 
q_sort(l, m - 1) ; q_sort(m + 1, u) ; 

} 

} 

mainO 
{ 

int i; randomize (); 

for (i = 0; i < N; i++) x[i] = random ( 100) ; 
q_sort(0, N - 1) ; 

for (i = 0; i < N; i++) cout « " " « x[i]; 

} 

The subroutine q_sort() has two parameters. Hence we must define a structure 

struct arg 
{ 

int left, right; 

}; 

for the parameters passing. We obtain the following text after the conversion: 

#include <windows.h> 
#include <stdlib.h> 
#include <time.h> 
#include <iostream.h> 
#include <conio.h> 
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#define N 100 
struct arg 
{ 

int left, right; 

>; 

int x [N] ; 

DWORD WINAPI q_sort(void* p) 
{ 

int l=((arg *)p)->left, u=((arg *)p)->right; 
int i , j ,m,t ,temp; 

if (Ku) 

{ 

t = x [1] ; m = 1 ; 

for (i=l+l; i<=u; i++) 

if (x[i]<t) 

{ 

m++; temp=x [m] ; x[m]=x[i]; x[i]=temp; 

} 

temp = x[l]; x[l]= x [m] ; x[m]=temp; 
arg *rl = new arg, *r2 = new arg; 
HANDLE H [2] ; 

rl->left=l; rl->right= m-1; 
r2->left= m+1; r2->right = u; 

H[0]= CreateThread(0,0,q_sort, (void *)rl,0,0); 
H[l]= CreateThread(0,0,q_sort, (void *)r2,0,0); 
for (i=0; i<2; i++) 

WaitForSingleObject(H[i] .INFINITE) ; 
delete rl; delete r2; 

} 

return 1 ; 

> 

void mainO 
{ 

int i ; 

randomize () ; 

for (i=0; i<N; i++) x [i] =random(100) ; 
for (i=0; i<N; i++) cout « " "« x[i]; 
cout « "\n"; 
arg a; 

a. left = 0; a. right = N-l; 
q_sort (&a) ; 

for (i=0; i<N; i++) cout « " "« x[i] ; 
getchO ; 
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The first call of the recursive function remains in the main program whereas the 
calls in q_sort() convert to CreateThreadQ. A project for this program must 
include the options "console application" and the "multithread application". 
Hence the creation of the project is given by the menu command File —> New — ► 
Project with Target Model "Console" and the option "Multithread". The files 
" .def ' and " .rc" must be deleted from the project. 

The program puts a sequence of 100 random numbers and then these num- 
bers are displayed in the undecreasing order. 

2 The pairing algorithm implementation 

It is possible the implementation of a multiply applied associative operation by 
the recursive subroutines. The subroutine has two parameters with numbers of 
first and last elements of an array: 

int sum(int 1, int r) // x [1] + ... + x[r] 
{ 

if(l == r) return x [1] ; 

else return sum(l, (l+r+l)/2 - 1) + sum( (1+r+l) /2 , r) ; 

} 

This subroutine is called in the main program by s=sum(0,n-l). Hence, we 
obtain by the conversion to the multithread application the following text: 

#include <windows.h> 
#include <stdlib.h> 
ttinclude <time.h> 
#include <iostream.h> 
#include <conio.h> 
#define N 32 
struct arg 
{ 

int 1 , r , rez ; 

}; 

int x [N] ; 

DWORD WINAPI sumCvoid* s) 
{ 

int i, l=((arg *)s)->l, r=((arg *)s)->r; 
if (l==r) ((arg *)s)->rez = x [1] ; 
else 
{ 

arg *rl = new arg, *r2 = new arg; 
HANDLE H [2] ; 

rl->l=l; rl->r= (l+r+l)/2-l; 

r2->l= (1+r+l) /2; r2->r = r; 

H[0]= CreateThread(0,0,sum, (void *)rl,0,0); 
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H[l]= CreateTnread(0,0,sum, (void *)r2,0,0); 
for (i=0; i<2; i++) 

WaitForSingleObject(H[i] , INFINITE) ; 
((arg *)s)->rez = (rl->rez)+(r2->rez) ; 
delete rl; delete r2; 

} 

return 1; 

} 

int sumOO 
{ 

int s=0, i; 

for (i=0; i<N; i++) s+=x[i]; 
return s; 

> 

void mainO 
{ 

int i ; 

randomize () ; 

for (i=0; i<N; i++) x [i] =random(100) ; 
arg t; 

t.l = 0; t.r = N-l; 
sum(fet) ; 

cout << "\n sum obtained = "<< t.rez; 
cout « "\n sum is = "« sumOO; 
getchO ; 

} 

Consider the pairing algorithm for the computing of the polynomial p{x) 
a + a\x + • • • + a n x n values by Horner's scheme: 

Po = a n ; Pi = xpo + fln-i; P2 = xpi + a„_ 2 ; ••• Pn — xp n -i + a . 

with p(x) = p n . If it possible to write pk = xpk-i + a n -k as the equation 

Pk \ _ ( x a„_ fe \ / p fe _i 

, 1 J \ o i A 1 

Therefore 

p(x) \ _ ( x a \ f x ai \ fx a„_i \ / a n 

i )-{o 1 ){o 1 )"\o 1 ){ 1 

and we can to compute the values of p(x) by the pairing algorithm for the matrix 
multiplications. 

Now we write the recursive subroutine of the matrix multiplication. The 
subroutine get parameters contained in a structure 

struct arg 
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{ 



int 1, r; 

double p, q; // Result 

>; 

The data are read from the external array a[i] and the variable x. We have a 
text of the subroutine: 

void prod(void *s) 
{ 

int i, 1 = ((arg*)s) -> 1, r = ((arg*)s) -> r; 

if(l == r) 

{ 

((arg*)s) -> p = x; 
((arg*)s) -> q = x [1] ; 

} 

else 
{ 

arg *rl = new arg, *r2 = new arg; 

rl -> 1 = 1; rl -> rl = (1 + r + l)/2 - 1; 

r2 -> 1 = (1 + r + l)/2; r2 -> r = r; 

((arg )s) -> p = (rl -> p) (r2 -> p) ; 

((arg )s) -> q = (rl -> p) (r2 -> q) + (rl -> q) ; 

delete rl; delete r2; 

} 



Converting this subroutine we obtain the following multithread application 
where the values do + a\x + a 2 x 2 + • • • + a^x N are calculated by the pairing of 
the Horner scheme 



} 



#include <windows.h> // 

#include <stdlib.h> // 

ttinclude <time.h> // 

#include <iostream.h> // 

#include <conio.h> // 

#include <dos.h> // 

#define N 20 // 

double x; // 

struct arg // 
{ 



multithread opportunity 
random numbers generation 
the first random number 
input /output 
for the getchO 
debugging by sleep () 
degree of the polynomial 
the value of the variable 
for the parameter settings 



int 1, r; // 



left and right numbers of 22-marix 
the first matrix string entries 



double p, q; // 



int a[N+l] ; // 
DWORD WINAPI prod(void* s)// 



coefficients of the polynomial 
the thread subroutine 
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int i, l=((arg *)s) -> 1, r=((arg *)s) -> r; 

sleep (random (2) ) ; // the delay for the debugging 

if (1 == r) 

{ 

((arg *)s) -> p = x; // the coefficient 
((arg *)s) ->q = a[r] ; // settings 

} 

else 
{ 

arg *rl = new arg, *r2 = new arg; 

HANDLE H[2]; // descriptors 

rl -> 1 = 1; rl -> r = (1 + r + l)/2 - 1;// boundaries 
r2 -> 1 = (1 + r + l)/2; r2 -> r = r; 
H[0] = CreateThread(0,0,prod, (void *)rl, 0,0);// create 
H[l] = CreateThread(0,0,prod, (void *)r2,0,0); // threads 
for (i = 0; i < 2; i++) // wait of 

WaitForSingleObject(H[i] .INFINITE) ; // terminates 

((arg *)s) -> p = (rl -> p)*(r2 -> p) ; 
((arg *)s) -> q = (rl -> p)*(r2 -> q)+(rl -> q) ; 
delete rl; delete r2; 

} 

return 1; 

} 

void mainO 
{ 

int i , j ; 

double w=0, v, z; 
randomize () ; 
for (i=0; i <=N; i++) 

a[i] = random (10) ; 
arg t; // parameter structure 

for (x = 0; x < 10; x += 1) // values of x 
{ 

t.l=0;t.r=N; // boundaries 

prod(fet) ; // call of the subroutine 

cout « "\n Polynomial value = " « t.q; // output 
z = 0; 

for (i = 0; i <= N; i++)// checking cycle 
{ 

v = 1; 

for (j = 0; j < i; j++) v = v*x; // computation of xi 
z += v*a[i] ; // the polynomial values 

} 

cout << " == "<< z; // displays for the checking 

getchO ; // wait of the keyword press 

} 
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> 



In fact the pairing algorithm computes in this application the product of 
N + 1 matricies. The first string entries of the product will be equal to x N+1 
and p * a[N] + q, where p and q are the first string entries of the product of N 
matrices. Thus the program displays the values p(x) = p * a[N] + q, and values 
obtained by the single thread program for the checking: 



Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 
Polynomial 



value 
value 
value 
value 
value 
value 
value 
value 
value 
value 



9 == 9 
53 == 53 
1.99675e+06 

4.74122e+09 
1.31457e+12 
1.07123e+14 
3.96656e+15 
8.47447e+16 
1.20753e+18 
1.26117e+19 



1.99675e+06 
4.74122e+09 
1.31457e+12 
1.07123e+14 
3.96656e+15 
8.47447e+16 
1.20753e+18 
1.26117e+19 



3 The synchronizing mechanism 

Suppose that a few threads work with one variable in the global memory. It 
is possible the access of one at most a thread to this variable in any time. 
The access synchronization of these threads is called a serialization problem. A 
solution of the serialization problem is based on a semaphore mechanism. A 
semaphore is the data structure which consists of one integer number s ^ and 
two functions P(s) and V(s) which are possible to unite in the following class: 

class Semaphore 
{ 

volative int s; //a semaphore counter 

public : 

Semaphore (int init) : s(init)-Q // initial counter value 
void P() {while (s==0) ; s — ;} // wait and locks 
void V() {s++;} // signal operation 

} 

Suppose that the functions of the semaphore class are realized by the hardware. 

Semaphores allow us solve the serialization problem. For example, consider 
the concurrent computing of the vector (xi, X2, ••• , x„) length square. We load 
n threads for the independent calculations of Xi * Xj. Every thread locks the 
global variable s after the calculation of d — Xi*Xi and yields the action s = s+d. 

We obtain the following program scheme: 

double x[n] ; // vector coordinates 
Semaphore s(l); // initialize s = 1 
double d = 0; // the global variable 
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main () 
{ 

/* here the threads are loaded ... */ 

} 

The thread subroutine can be contains the following operators: 

double y = x[i]*x[i]; 

s.P(); // the lockout of d resource 

d = d + y; 

s.V(); // the signal operation 

Our aim is to write this program in the Borland C/C++. We will be use the 
following subroutines for the semaphore operations. The part of the constructor 
is belong to the function: 

HANDLE s = CreateSemaphore (NULL, i, n, NULL); 

where i is the initial value of the semaphore counter whereas n is the maximal 
value. The role of the call s.PQ is played by 

WaitForSingleObject(s, INFINITE) ; 

and there call s.V() is executed by 

ReleaseSemaphore (s , 1, NULL). 

These subroutines allow us to write the following solution of the serialization 
problem: 

#include <windows.h> 
#include <stdlib.h> 
#include <time.h> 
#include <iostream.h> 
#include <conio.h> 
#define n 100 
volatile int j=0; 
HANDLE mut; 
double d=0; 
double x [n] ; 

DWORD WINAPI Thr(LPV0ID v) 
{ 

double *w = (double *)v; double u=(*w)*(*w); 
WaitForSingleObject(mut, INFINITE) ; 
d= d+u; 

ReleaseSemaphore (mut , 1 , NULL) ; 
return 1; 

} 

double sumOO 
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double s0=0; int k; 
for (k=0; k<n; k++) 

s0+= x [k] *x [k] ; 
return sO; 

} 

void mainO 
{ 

int i ; 

randomize () ; 

for (i=0; i<n; i++) x[i]=(0.+random(100))/100+5; 
mut = CreateSemaphore (NULL, 1 , 1 , NULL); 
for (i=0; i<n; i++) 
{ 

CreateThread(NULL,0,Thr, (void *) (&x[i] ) ,0,0) ; 

} 

while (j<n) ; 

cout « "\nValue obtained by threads = "« d; 

cout << "\nValue computed by function = "<< sumOO; 

getchO ; 



4 A channel class for the producer and con- 
sumer problem 

Using the idea the Occam we define a class which objects are channels. Each 
channel is a data queue organized as an array. Suppose that this array contains 
the double precision numbers. The access to the queue is provided by the 
overload operations << for the writing and >> for the reading. Using a classical 
solution we define the class of the channel. We write this definition into the file 
channel. h: 

// channel. h 
template<class Type> 
class channel 
{ 

Type *buf ; 
int size; 
HANDLE s, 

empty, 

full; 

int countr, countw; 
public : 

channel (int n) : size(n) 
{ 



// the buffer for a queue 

// the buffer size 

// the access semaphore 

// number of free places in the queue 

// number of full places in the queue 

// read/write counters 

// constructor 
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buf = new Type [n] ; // the memory of the buffer 

countr=0 ; countw=0 ; 

s=CreateSemaphore (NULL, 1 , 1 ,NULL) ; // semaphore initialization 
empty=CreateSemaphore (NULL, n,n, NULL) ; // n free places 
f ull=CreateSemaphore (NULL, 0,n, NULL) ; // no full places 

} 

void operator « (Type d) // write to channel 

{ 

WaitForSingleObject (empty, INFINITE); // wait free places 
WaitForSingleObject(s, INFINITE); // take buffer 

buf [countw++] =d; // write to queue 

if (countw==size) countw=0; 
ReleaseSemaphore (s , 1 ,NULL) ; 

ReleaseSemaphore (f ull, 1, NULL) ; // full++ 

} 

void operator » (Typefe d) // read from channel 
{ 

WaitForSingleObject (full, INFINITE); // wait of data 
WaitForSingleObject(s, INFINITE); // take buffer 
d = buf [countr] ; countr++; // read 

if (countw==size) countw=0; 
ReleaseSemaphore (s , 1 ,NULL) ; 

ReleaseSemaphore (empty , 1 ,NULL) ; // empty++ 

} 

}; 

It is a generalization with the template type instead of double. Now we have 
the following solution of the producer and consumer problem: 

#include <windows.h> 
#include <stdlib.h> 
#include <time.h> 
#include <iostream.h> 
#include <conio.h> 
#include " channel. h" 
double out [20] ; 

DWORD WINAPI producer (LPVOID v) 
{ 

int j; 
double dl ; 

for (j=0; j<20; j++) 

{ 

dl = (double) (1+j) ; 

* (channel<double>*) v<<dl ; 

} 

return 1; 
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DWORD WINAPI consumer (LPVOID v) 
{ 

int k; 

for (k=0; k<20; k++) 
{ 

*(channel<double>*)v»out [k] ; 

} 

return 1 ; 

> 

void mainO 
{ 

channel<double> c0(10); 
int i=0; 

CreateThread (NULL, 0, producer, (LPVOID) &c0,0,0); 
CreateThread (NULL, 0, consumer, (LPVOID) &c0,0,0); 
cout«"\n"; f or(i=0; i<20; i++) cout « " "« out [i] ; 
getchO ; 

cout«"\n"; for(i=0;i<20;i++) cout « " "« out [i] ; 
getchO ; 

} 

Here the producer writes to the channel the sequence of numbers. The 
consumer reads this sequence and writes it to the array out[]. The application 
displays 20 values of 0, and the sequence 1, 2, • •, 20 after the key input. 

5 Wave systems and their Petri nets 

A wave system is the generalization of the pipeline. Consider the computing of 
the values y n = f(g(x n )) with n = 0, 1, 2, • • •. It is possible to load the threads 
which states are described by the following Petri net: 




Pi P2 P3 



where the main program M and threads Tl, T2, T3 are executed concurrently. 
The main program generated random numbers x n and writes x n to the channel 
corresponding to the channel p\. The thread Tl computes g(x n ) and writes it 
to the channel pi. The thread T2 computes f(g(x n )) and the thread T3 gets 
the data from the channel pz and puts to the display. Such a system is called 
the pipeline and has the following generalization. For example, consider the 
computation of values 

x„ = u n + sin(vl), y n = exp(sin(u n - v n )). 
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Applicate the flow control with the channel communications. The factorization 
of these operations leads us to the 10 threads executed concurrently. The threads 
correspond to the transitions of the Petri net which is shown in Figure 

The Petri net consists of 11 places corresponding to the channels and 10 
transitions corresponding to the threads. Input of (u n ,v n ) and output of (x n , y n ) 
will be yield in the main program. Thus the computations will be yield in the 
six threads. The array of channels 

Channel c[ll] = {10,10,10,10, ... ,10}; 

is declared in the main program. Each channel is a queue of 10 elements of the 
array of double precision numbers. These channels connect by global pointer 
declared as channel *ps[l\\. The threads execute concurrently. Each of threads 
waits an input data and then computes an operation. For example, the subrou- 
tine mult() gets the data from the channels c[0] and c[l] and sends their product 
to the channel c[4]. 

#include <windows.h> 
#include <stdlib.h> 
#include <time.h> 
#include <iostream.h> 
#include <conio.h> 
#include <math.h> 
#include " channel. h" 
#define n 20 
channel *pc [11] ; 
DWORD WINAPI mult(LPVOID) 
{ 

int j; 

double dl, d2; 
for (j=0; j<n; j++) 
{ 

*pc[0]»dl; *pc[l]»d2; 
*pc[4]«(dl*d2) ; 

} 

return 1 ; 

} 

DWORD WINAPI minus (LPV0 ID) 
{ 

int j; 

double dl, d2; 

for (j=0; j<n; j++) 
{ 

*pc[2]»dl; *pc[3]»d2; 
*pc[5]«(dl-d2) ; 

} 

return 1 ; 



// multiplication thread 



// get data from channels and 1 
// multiply and send to 4 



// subtract thread 



// get data from channels 2 and 3 
// subtract and send to 5 
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> 

DWORD WINAPI sinus (LPVOID) 
{ 

int j; 
double dl ; 

for (j=0; j<n; j++) 

{ 

*pc [4]»dl; 

*pc [7]«sin(dl) ; 

} 

return 1 ; 

} 

DWORD WINAPI sinus2 (LPVOID) 
{ 

int j; 
double dl ; 

for (j=0; j<n; j++) 

{ 

*pc [5]»dl; 
*pc[8]«sin(dl) ; 

} 

return 1; 

} 

DWORD WINAPI plus (LPVOID) 
{ 

int j; 

double dl, d2; 

for (j=0; j<n; j++) 
{ 

*pc[6]»dl; *pc[7]»d2; 
*pc[9]«(dl+d2) ; 

} 

return 1; 

> 

DWORD WINAPI expo (LPVOID) 
{ 

int j; 
double dl ; 

for (j=0; j<n; j++) 

{ 

*pc [8]»dl; 
*pc[10]«exp(dl) ; 

} 

return 1 ; 

} 

void mainO 



// sinus thread 



// get from 4 
// send to 7 



// the second thread for sinus 



// get from 5 
// send to 8 



// thread for add 



// get from 6 and 7 
// send to 9 



// thread for the exponent 



// get from 8 
// send to 10 



// main program 
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{ 

channel c [11] ={10 , 10 , 10 , 10 , 10 , 10 , 10 , 10 , 10 , 10 , 10} ; 
// initialize threads 

int i; // by 10 elements 

for (i=0; i<ll;i++) pc [i] = &c [i] ; // pointers to channels 

CreateThread (NULL, 0, mult, 0, 0, 0) ; // load the threads 

CreateThread (NULL , , minus ,0,0,0); 

CreateThread (NULL , , sinus ,0,0,0); 

CreateThread (NULL , , sinus2 ,0,0,0); 

CreateThread (NULL , , plus ,0,0,0); 

CreateThread (NULL, 0, expo, 0,0,0); 
double u,v,x,y; 

for (i=0; i<n; i++) 

{ 

u=i*i; v=i+l; 

c[0]«v; c[l]«v; // send to channels 0,1,2,3,6 

c[2]«u; c[3]«v; c[6]«u; 

} 

for (i=0; i<n; i++) 
{ 

c[9]»x; c[10]»y; cout«"\n"«x; 

cout<<"=="; // read and write the results 

cout<< (double) (i*i+sin(0.+(i+l)*(i+l))) ; // comparison 
cout«" "«y«"=="«(double) (exp(sin(i*i-i-l)) ) ; 

} 

getch() ; 

} 

The application displays: 



0.841471 == 0.841741 0.431076 == 0.431076 
0.243198 == 0.243198 0.431076 == 0.431076 



4.41212 


== 4.41212 




2.31978 = 


= 2.31278 


8.7121 = 


== 8.7121 





.383305 == 


0.383305 


15.8676 


== 15.8676 




0.367883 


== 0.367883 


24.0082 


== 24.0822 




1.16169 = 


= 1.16169 


35.0462 


== 35.0462 




0.514977 


== 0.514977 


49.92 == 


=49.92 0. 


853318 == 


.853318 


63.3701 


== 63.3701 




0.36797 = 


= 0.36797 


80.4936 


== 80.4936 




2.58844 = 


= 2.58844 


100.999 


== 100.999 




2.36332 = 


= 2.36332 


120.509 


== 120.509 




2.26312 = 


= 2.26312 


143.398 


== 143.398 




0.444145 


== 0.444145 


169.94 = 


== 169.94 





.417556 == 


0.417556 


195.07 = 


== 195.07 





.392016 == 


0.392016 


224.001 


== 224.001 




2.70869 = 


= 2.70869 


255.973 


== 255.973 




1.26705 = 


= 1.26705 
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288.596 == 288.596 2.08175 == 2.08175 
324.279 == 324.279 0.7692 == 0.7692 
360.149 == 360.149 2.69286 == 2.69286 



The comparison of the results shows that the wave system is well defined. 
We conclude that there is the well opportunity for the study of the concurrent 
algorithms on the every personal computer. 
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